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(54) Writing guide for a free-form document editor 



(57) Displaying a writing guide for a free-form doc- 
ument editor by detecting electronic ink, classifying the 
detected ink as corresponding to handwriting or a draw- 
ing, and displaying a handwriting guide or a drawing 
guide in response to classifying the detected ink as 
handwriting or a drawing, respectively. The writing 



guides can provide feedback regarding the classifica- 
tion of the detected ink and can be resized as additional 
ink is detected. The handwriting guide can comprise a 
writing area, writing guide portion, bullet guide, indent 
marker, character notches, gesture guide, or sacrificial 
guide to create meaningful structure within an outline 
object of an electronic document. 
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Description 

RELATED PATENT APPLICATIONS 

[0001] This patent application Is related to the follow- 
ing co-pending, non-provlslonal patent applications: U. 
S. Patent Application Serial No. 09/852,799 filed on May 
1 1 , 2001 , entitled "Serial Storage of Ink and Its Proper- 
ties" and U.S. Patent Application Serial No. 09/430,150 
filed on October 29, 1999, entitled "Universal File For- 
mat for Digital Rich Ink Data." Both related applications 
are assigned to the IVIicrosoft Corporation. 
[0002] This patent application also is related to the fol- 
lowing non-provisional patent applications: U.S. Patent 

Application Serial No. , entitled 

"Method and System for Editing Electronic Ink," having 
attorney docket number 06576. 1 05074; U.S. Patent Ap- 
plication Serial No. , entitled "Meth- 
od and System for Selecting Objects on a Display De- 
vice," having attorney docket number 06576.105075; U. 
S. Patent Application Serial No. , en- 
titled "Resolving Document Object Collisions," having 
attorney docket number 06576.1 05076; U.S. Patent Ap- 
plication Serial No. , entitled "Space 

Management for Electronic Documents," having attor- 
ney docket number 06576.105077; U.S. Patent Appli- 
cation Serial No. , entitled "System 

and Method for Automatically Recognizing Electronic 
Handwriting in an Electronic Document and Converting 
to Text," having attorney docket number 06576.1 05078; 

U.S. Patent Application Serial No. , 

entitled "Method and System for Designating Data Ob- 
jects with Designation Tools," having attorney docket 
number 06576.1 05079; and U.S. Patent Application Se- 
rial No. , entitled "Method and Sys- 
tem for Displaying and Linking Ink Objects with Recog- 
nized Text and Objects," having attorney docket number 
06576.105080. Those applications and the application 
herein are all being filed on the same date, June 28, 
2002, and are assigned to the Microsoft Corporation. 
[0003] The subject matter of each related application 
Identified above Is fully Incorporated herein by refer- 
ence. 

FIELD OF THE INVENTION 

[0004] The present invention relates to creating elec- 
tronic documents using a free-form document editor. 
Particularly, the present invention relates to writing 
guides that provide a user interface in a free-form doc- 
ument editor. 

BACKGROUND OF THE INVENTION 

[0005] One of the simplest methods of recording and 
communicating Information Is the traditional method of 
writing the Information on a piece of paper with a writing 
Instrument such as a pen. Writing Information by hand 



2 

on a piece of paper is inexpensive and can be done 
quickly and easily with little preparation. The traditional 
method is also flexible in that a writer generally can write 
In any format anywhere on the page. One of the llmlta- 
5 tlons with handwritten work Is that It Is not easily manip- 
ulated or transferred to other contexts. In other words, 
changing or transferring a piece of handwritten script 
typically requires rewriting the script on another medi- 

10 [0006] With the widespread use of personal comput- 
ers, textual information often is recorded using word 
processing software running on a personal computer. 
The advantage of such electronic methods of recording 
information is that the information can be stored easily 

'5 and transferred to other remote computing devices and 
electronic media. Such electronically recorded text also 
can be easily corrected, modified, and manipulated in a 
variety of different ways. 

[0007] Typical computer systems, especially compu- 
te ter systems using graphical user interface (GUI) sys- 
tems such as Microsoft WINDOWS, are optimized for 
accepting user input from one or more discrete input de- 
vices. Common input devices include a keyboard for en- 
tering text and a pointing device, such as a mouse with 
25 one or more buttons, for controlling the user interface. 
The keyboard and mouse interface facilitates creation 
and modification of electronic documents including text, 
spreadsheets, database fields, drawings, and photos. 
[0008] One of the limitations with conventional GUI 
30 systems Is that a user must generally type the text they 
are entering Into the personal computer using the key- 
board. Entering text using a keyboard can be slower and 
more cumbersome than handwriting. Although recent 
advances have been made in reducing the size of per- 
35 sonal computers, they are still not as portable and easily 
accessible as traditional pen and paper. Furthermore, 
traditional pen and paper provide the user with consid- 
erable flexibility for editing a document, recording notes 
in the margin, and drawing figures and shapes. In some 
40 instances, a user may prefer to use a pen to mark-up a 
document rather than to review the document on- 
screen, because of the ability to freely make notes out- 
side of the confines of the keyboard and mouse inter- 
face. 

45 [0009] To address the shortcomings of traditional key- 
board and mouse interfaces, there have been various 
attempts to create an electronic tablet that can record 
handwriting. The electronic tablets typically comprise a 
free-form document editor for creating free-form docu- 

50 ments, which is considered a valuable tool for enabling 
a user to engage in free-form note taking. The electronic 
tablets typically also comprise a screen and a handheld 
device that is similar to a pen. A user can write with the 
handheld device on the screen of the electronic tablet 

55 in a similar manner to traditional pen and paper. The 
electronic tablet can "read" the strokes of the user's 
handwriting with the handheld device and recreate the 
handwriting in electronic form on the screen with "elec- 
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tronic ink. " That electronic tablet approach can be em- 
ployed in a variety of ways, including on a personal com- 
puter and on a handheld computing device. 
[0010] Recently, other computer systems have been 
developed that provide for more sophisticated creation 
and manipulation of electronic ink. Those applications 
(for example, drawing applications associated with 
Palm and PocketPC operating systems) can permit the 
capturing and storing of drawings. Those drawings in- 
clude other properties associated with the ink strokes 
used to make up the drawings. For instance, line width 
and color may be stored with the ink. One goal of these 
systems is to replicate the look and feel of physical ink 
being applied to a piece of paper. However, physical ink 
on paper can have significant amounts of information 
not captured by the electronic collection of coordinates 
and connecting line segments. Some of this Information 
may include outline structure and type of writing (such 
as handwriting or drawing). 

[0011] In spite of the advances in electronic tablets 
and electronic ink, there are still several limitations with 
the performance of such electronic handwriting devices. 
For example, conventional free-form document editors 
cannot recognize document structure as the user inputs 
inkstrokes on the tablet. Each line of handwriting is con- 
verted separately to text and aligned along a left margin 
of the screen. Accordingly, any indented formatting pro- 
vided by the user for the inkstrokes Is lost on conversion 
to text. 

[0012] Another problem with conventional free-form 
document editors is the blind conversion of ink to text. 
For example, ink can be converted to text and located 
in the screen in the exact location of the input ink 

strokes. However, that method merely provides elec- 
tronic text without linking the text throughout the elec- 
tronic document. Accordingly, the user cannot revise 
and edit the electronic text when it is converted in that 
manner. Additionally, the text cannot always be export- 
ed to other application programs such as a word proc- 
essor application program. 

[0013] Another problem with conventional free-form 
document editors is a lack of feedback regarding the 
type of ink input by the user. The types of ink include 
handwriting and drawing. The editor may erroneously 
classify handwriting as a drawing. However, the editor 
cannot indicate the classification to the user. According- 
ly, the user does not know of the error until the user re- 
alizes that the ink was not converted to text. By that time, 
many lines of Ink may be erroneously classified, causing 
excess work by the user to correct the formatting. 
[0014] Accordingly, there is need in the art for improv- 
ing handwriting recognition and conversion in free-form 
electronic documents. A further need exists for provid- 
ing feedback regarding the free-form document editor's 
classification of handwritten ink strokes. Additionally, a 
need exists in the art for guiding the user's ink input to 
increase text recognition accuracy. A need also exists 
for allowing a user to input structured formatting for 



handwriting and for converting the formatting into text 
usable by other application programs. 

SUMiVIARY OF THE INVENTION 

5 

[0015] The present invention can provide a writing 
guide as a user interface for a free-form document edi- 
tor. The writing guide can inform a user of the editor's 
classification of ink strokes input by the user and can 

10 allow the user to create structured notes within an out- 
line object of the free-form document editor. Additionally, 
the writing guide can guide a user in building meaningful 
electronic document structure that can be converted to 
text and repurposed in other applications with high ac- 

'5 curacy. 

[0016] The writing guide can provide a user Interface 
for creating and extending lines In an outline object. The 
writing guide can provide visual feedback to both the us- 
er and the document editor, confirming that the ink is its 
20 own line of handwriting or that the ink is attached to an 
existing line of handwriting. Additionally, the writing 
guide can help the user to write in a more controlled 
fashion. For example, following the writing guide can re- 
sult in writing in straight lines and not at arbitrary angles. 
25 Accordingly, the user can create handwriting having a 
high degree of recognition. 

[0017] The present invention can provide two main 
types of writing guides, a drawing writing guide and a 
handwriting writing guide (hereinafter a drawing guide 
30 and a handwriting guide, respectively). A writing guide 
can appear on the document editor page as the user 
begins to input electronic ink strokes onto a tablet. If a 
writing guide module determines that the ink strokes 
comprise handwriting writing, then a handwriting guide 
35 can be displayed. As the user continues to write and to 
create more structure, various forms of the handwriting 
guide can allow the user to create more complex struc- 
ture within the outline object. For example, writing 
guides, writing guide portions, bullet writing guides, in- 
40 dent markers, and other writing guides can show the us- 
er how to predictably create more complicated and 
meaningful structure within an outline object. As the out- 
line object increases, the writing guides can indicate the 
size and shape of each paragraph and can change size 
45 to indicate indent levels and multiple line paragraphs. 
[0018] If the writing guide module determines that the 
ink strokes comprise a drawing, then a drawing guide 
can be displayed. The drawing guide can allow multiple 
ink strokes to be grouped together as a drawing. The 
50 drawing guide can indicate that classification to the user 
and can inform the document editor not to convert the 
ink strokes to text. 

[0019] These and other aspects, objects, and fea- 
tures of the present invention will become apparent from 
55 the following detailed description of the exemplary em- 
bodiments, read in conjunction with, and reference to, 
the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Figure 1 is a block diagram illustrating an exemplary 
operating environment for implementation of vari- 
ous embodiments of the present invention. 
Figure 2 is a block diagram depicting the primary 
functional components of an exemplary free-form 
document editor and related input devices. 
Figure 3 Is a flow chart depicting a method for dis- 
playing a writing guide according to an exemplary 
embodiment of the present invention. 
Figure 4 Illustrates a writing guide displayed on a 
tablet according to an exemplary embodiment of the 
present invention. 

Figure 5 is a flow chart depicting a method for clas- 
sifying detected ink as a drawing or handwriting ac- 
cording to an exemplary embodiment of the present 
invention. 

Figure 6 Is a flow chart depicting a method for dis- 
playing a handwriting guide according to an exem- 
plary embodiment of the present Invention. 
Figure 7A illustrates dimensions of a handwriting 
guide according to an exemplary embodiment of the 
present invention. 

Figure 7B illustrates dimensions of a handwriting 
guide according to another exemplary embodiment 
of the present invention. 

Figure 8 Is a flow chart depicting a method for es- 
tablishing a baseline of a writing guide according to 
an exemplary embodiment of the present invention. 
Figure 9 illustrates a representative ink histogram 

showing the amount of ink In respective segments 
of a line of ink. 

Figure 1 0 is a flowchart depicting a method for es- 
tablishing the baseline of a writing guide according 
to an alternative exemplary embodiment of the 
present Invention. 

Figure 11 Illustrates dimensions of a writing guide 
established by the method of Figure 10 according 
to an alternative exemplary embodiment of the 
present invention. 

Figure 12 is a flowchart depicting a method for re- 
calculating handwriting guide dimensions accord- 
ing to an exemplary embodiment of the present in- 
vention. 

Figure 1 3 Illustrates a structured outline handwriting 
guide for an Indented outline structure according to 
an exemplary embodiment of the present Invention. 
Figures 14A, 14B, 14C, 14D, and 14E Illustrate an 
operation of a structured outline handwriting guide 
according to an exemplary embodiment of the 
present Invention. 

Figure 15 Illustrates a language handwriting guide 
for languages without spaces between their char- 
acters according to an exemplary embodiment of 
the present invention. 



Figure 1 6 illustrates agesture handwriting guide ac- 
cording to an exemplary embodiment of the present 
invention. 

Figure 1 7 Is a flow chart depicting a method for ac- 
tivating a gesture handwriting guide according to an 
exemplary embodiment of the present invention. 
Figure 18 illustrates a sacrificial writing guide ac- 
cording to an exemplary embodiment of the present 
invention. 

Figure 19 Is a flow chart depicting a method for ac- 
tivating a sacrificial writing guide according to an ex- 
emplary embodiment of the present Invention. 
Figure 20 illustrates active writing guide portions for 
adding handwriting to existing object elements ac- 
cording to an exemplary embodiment of the present 
invention. 

Figure 21 is a flow chart depicting a method for clas- 
sifying off-guide writing according to an exemplary 
embodiment of the present invention. 
Figure 22A illustrates an off-guide ink stroke repre- 
senting an edit according to an exemplary embod- 
iment of the present Invention. 
Figure 22B illustrates a group of off-guide Ink 
strokes representing an annotation according to an 
exemplary embodiment of the present invention. 
Figures 23A and 238 Illustrate a nested heading 
writing guide according to an exemplary embodi- 
ment of the present Invention. 
Figure 24 Illustrates a drawing writing guide accord- 
ing to an exemplary embodiment of the present in- 
vention. 

Figure 25 illustrates a method for displaying a draw- 
ing guide according to an exemplary embodiment 
of the present invention. 

Figure 26 Is a flowchart depicting a method for re- 
calculating drawing guide dimensions according to 
an exemplary embodiment of the present Invention. 

DETAILED DESCRIPTION OF THE EXEMPLARY 
EMBODIIVIENTS 



[0021] The present invention can provide a writing 
guide as a user interface for a free-form document editor 
to allow creation of an electronic document from hand- 

45 written electronic ink. The writing guide can indicate to 
a user whether the editor has classified ink strokes as 
handwriting or a drawing. For handwriting, the present 
invention can present a handwriting guide to the user. 
The handwriting guide can provide detailed structural In- 

50 formation regarding the format structure of the electron- 
ic document. For example, the handwriting guide can 
indicate bullets and Indent levels for Input handwriting. 
Additionally, the handwriting guide can Indicate whether 
multiple lines are formatted as one paragraph or as in- 

55 dividual lines. For a group of ink strokes classified as a 
drawing, a drawing guide can be presented. The draw- 
ing guide can allow the ink strokes to be formatted and 
manipulated as a group. 
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[0022] Exemplary embodiments will be described 
generally in the context of software modules running in 
a computing environment. The processes operations 
performed by the software modules include the manip- 
ulation of signals by a client or server and the mainte- 
nance of those signals within data structures resident in 
one or more of local or remote memory storage devices. 
Such data structures impose a physical organization up- 
on the coiiection of data stored within a memory storage 
device and represent specific electrical or magnetic el- 
ements. Those symbolic representations are the means 
used by those skilled in the art of computer program- 
ming and computer construction to effectively convey 
teachings and discoveries to others skilled in the art. 
[0023] The present invention also includes a compu- 
ter program that embodies the functions described here- 
in and illustrated in the appended flow charts. However, 
it should be apparent that there could be many different 
ways of implementing the invention in computer pro- 
gramming, and the invention should not be construed 
as limited to any one set of computer program instruc- 
tions. Further, a skilled programmer would be able to 
write such a computer program to implement the dis- 
closed invention based on the flow charts and associat- 
ed description in the application text. Therefore, disclo- 
sure of a particular set of program code instructions is 
not considered necessary for an adequate understand- 
ing of how to make and use the invention. The inventive 
functionality of the claimed computer program will be ex- 
plained in more detail in the following description read 
in conjunction with the Figures illustrating the program 
flow. 

[0024] Referring now to the drawings, in which like nu- 
merals represent like elements, aspects of the present 
invention and exemplary operating environment will be 
described. 

An Exemplary Operating Environment 

[0025] Figure 1 illustrates an exemplary operating en- 
vironment for implementation of the present invention. 
The exemplary operating environment includes a gen- 
eral-purpose computing device in the form of a conven- 
tional personal computer 120. Generally, the personal 
computer 1 20 includes a processing unit 1 21 , a system 
memory 122, and a system bus 123 that couples various 
system components including the system memory 122 
to the processing unit 1 21 . The system bus 1 23 may be 
any of several types of bus structures including a mem- 
ory bus or memory controller, a peripheral bus, and a 
local bus using any of a variety of bus architectures. The 
system memory includes a read-only memory (ROM) 
124 and a random access memory (RAM) 125. A basic 
input/output system (BIOS) 126, containing the basic 
routines that help to transfer information between ele- 
ments within the personal computer 1 20, such as during 
start-up, is stored in the ROIVI 124. 
[0026] The personal computer 120 further includes a 



hard disk drive 1 27 for reading from and writing to a hard 
disk, not shown, a magnetic disk drive 128 for reading 
from or writing to a removable magnetic disk 129, and 
an optical disk drive 130 for reading from or writing to a 
5 removable optical disk 131 such as a CD-ROIVI, DVD, 
or other optical media. The hard disk drive 127, mag- 
netic disk drive 128, and optical disk drive 130 are con- 
nected to the system bus 1 23 by a hard disk drive inter- 
face 132, a magnetic disk drive interface 133, and an 
'0 optical disk drive interface 134, respectively. Although 
the exemplary environment described herein employs 
the hard disk 127, removable magnetic disk 129, and 
removable optical disk 131, it should be appreciated by 
those skilled in the art that other types of computer read- 
's able media which can store data that is accessible by a 
computer, such as magnetic cassettes, flash memory 
cards, digital video disks, Bernoulli cartridges, RAMs, 
ROMs, and the like, may also be used in the exemplary 
operating environment. The drives and their associated 
20 computer readable media provide nonvolatile storage of 
computer-executable instructions, data structures, pro- 
gram modules, and other data for the personal computer 
120. 

[0027] A number of program modules may be stored 
25 on the hard disk 127, magnetic disk 129, optical disk 
131, ROM 1 24, or RAM 1 25, including an operating sys- 
tem 135, a document editor 136, an ink processor 138, 
and an electronic document 139. Program modules in- 
clude routines, sub-routines, programs, objects, compo- 
30 nents, data structures, etc., which perform particular 
tasks or implement particular abstract data types. As- 
pects of the present invention may be implemented in 
the form of a document editor 136 that can operate in 
concert with the ink processor 1 38 to edit the electronic 
35 document 139. Thedocument editor 136generallycom- 
prises computer-executable instructions for creating 
and editing electronic documents. The ink processor 
138 is generally accessible to the document editor 136, 
but also can be implemented as an integral part of the 
40 document editor 136. 

[0028] A user may enter commands and information 
into the personal computer 120 through input devices, 
such as a keyboard 140 and a pointing device 142. 
Pointing devices may include a mouse, a trackball, and 
45 a stylus or an electronic pen that can be used in con- 
junction with an electronic tablet. Other input devices 
(not shown) may include a microphone, joystick, game 
pad, satellite dish, scanner, or the like. Those and other 
input devices are often connected to the processing unit 
50 122 through a serial port interface 146 that is coupled 
to the system bus 123, but may be connected by other 
interfaces, such as a parallel port, game port, a universal 
serial bus (USB), or the like. A display device 147 may 
also be connected to the system bus 123 via an inter- 
55 face, such as a video adapter 148. In addition to the 
monitor, personal computers typically include other pe- 
ripheral output devices (not shown), such as speakers 
and printers. 
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[0029] The personal computer 120 may operate in a 
networked environment using logical connections to 
one or more remote computers 149. A remote computer 
149 may be another personal computer, a server, a cli- 
ent, a router, a network PC, a peer device, or other com- 
mon network node. While a remote computer 149 typi- 
cally includes many or all of the elements described 
above relative to the personal computer 120, only a 
memory storage device 150 has been illustrated in Fig- 
ure 1. The logical connections depicted in Figure 1 in- 
clude a local area network (LAN) 151 and a wide area 
network (WAN) 1 52. Such networking environments are 
commonplace in offices, enterprise-wide computer net- 
works, intranets, and the Internet. 
[0030] When used in a LAN networking environment, 
the personal computer 120 is often connected to the lo- 
cal area network 151 through a network interface or 
adapter 153. When used in a WAN networking environ- 
ment, the personal computer 120 typically includes a 
modem 154 or other means for establishing communi- 
cations over the WAN 152, such as the Internet. The 
modem 1 54, which may be internal or external, is con- 
nected to the system bus 1 23 via the serial port interface 
146. In a networked environment, program modules de- 
picted relative to the personal computer 1 20, or portions 
thereof, may be stored in the remote memory storage 
device 1 50. It will be appreciated that the network con- 
nections shown are exemplary and other means of es- 
tablishing a communications link between the comput- 
ers may be used. 

[0031] Moreover, those skilled in the art will appreci- 
ate that the present invention may be implemented in 
other computer system configurations, including hand- 
held devices, multiprocessor systems, microprocessor 
based or programmable consumer electronics, network 
personal computers, minicomputers, mainframe com- 
puters, and the like. The invention may also be practiced 
in distributed computing environments, where tasks are 
performed by remote processing devices that are linked 
through a communications network. In a distributed 
computing environment, program modules may be lo- 
cated in both local and remote memory storage devices. 
[0032] Figure 2 is a block diagram depicting the pri- 
mary functional components of an exemplary free-form 
document editor and related input devices. Specifically, 
Figure 2 depicts typical hardware and software compo- 
nents used in operating an exemplary embodiment of 
the invention in the context of a free-form document ed- 
itor. Conventional input devices are represented by the 
keyboard 260 and the pointing device 265 (e.g., mouse, 
pen or stylus, and tablet). A user can enter commands 
and information into the computer 1 20 using the pointer 
device and/or another input device. Other input devices 
(not shown) may include a microphone, joystick, game 
pad, satellite dish, and scanner. A conventional output 
device, such as monitor 270, is used to display informa- 
tion from the computer 120. Other output devices (not 
shown) can include a printer or speaker. 



10 

[0033] Other hardware components shown in Figure 
2 include an electronic tablet 250 and an accompanying 
stylus 255. The tablet 250 and stylus 255 are used to 
Input handwriting strokes which can be converted to da- 
5 ta, referred to as electronic Ink. The electronic Ink may 
be incorporated into an electronic document 220 and 
may be displayed on either the electronic tablet 250, the 
monitor 270, or both. Although the electronic tablet 250 
and the monitor 270 are illustrated as being distinct, in 
• 0 an exemplary embodimentof the present invention, they 
can be integrated into a single component. The joint tab- 
let/monitor component has the ability to display informa- 
tion and receive input from the stylus 255. 
[0034] In the representative architecture 200, all of the 
'5 hardware components are coupled to an ink processing 
software module 225. The ink processing module 225 
Is operable to receive data from the electron Ic tablet 250 
and to render that data as electronic ink. In one embod- 
iment of the present invention, the ink processing mod- 
20 ule225can be a collection of software modules that per- 
form different tasks for rendering handwriting strokes as 
electronic ink. For example, the stylus and ink module 
228 can receive data describing the positions and an- 
gles of the stylus for a series of handwriting strokes. The 
25 stylus and ink module 228 can interpret the data for ren- 
dering electronic ink. Other software modules, such as 
a gesture recognizer 230 and a text recognition module 
235 can be designed to identify certain handwriting 
strokes and assign them a particular significance. For 
30 example, certain gestures such as a cross-out may be 
recognized and associated with other editing process- 
es. The ink processing module 225 also can include an 
erasing functions module 240 for removing electronic 
ink that has been previously rendered as text. 
35 [0035] Although ink processing modules are known in 
the art and necessary for an electronic tablet to function, 
a novel document editing module has been developed 
by the Microsoft Corporation of Redmond, Washington 
that capitalizes on the benefits of handwriting process- 
10 ing technology. The exemplary document editing mod- 
ule (i.e., document editor) isafree-form document editor 
that leverages the functionality of electronic handwriting 
technology to enable more effective and efficient note 
taking. Specifically, a document editing module 205 fa- 
45 cilitates the manipulation of electronic ink so that a user 
can create and modify an electronic document 220 with 
greater ease and sophistication. In an exemplary em- 
bodiment, the document editing module 205 comprises 
a collection of software modules for controlling and ma- 
50 nipulating electronic ink rendered on the monitor 270. 
For example, a parsing module 210 can be used to iden- 
tify handwriting strokes that are selected by the user for 
editing. Selected strokes may by highlighted or shaded 
to assist the user in identifying which strokes are to be 
55 edited. A classifier module 215 can identify certain 
handwriting strokes as being part of a word or drawing 
and may designate document objects accordingly. Soft- 
ware modules such as the layout module 216 and the 
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insert space module 21 7 can be designed to control how 
electronic ink is rendered and moved. A writing guide 
module 227 can provide a writing guide according to an 
exemplary embodiment of the present Invention on the 
tablet 250. 

[0036] Exemplary embodiments of the present Inven- 
tion relate most directly to providing a writing guide on 
the tablet 250. The functionality of various embodiments 
may be performed by the writing guide module 227 or 
may be fully integrated into the operation of the docu- 
ment editing module 205. Figures 1 and 2 provide a gen- 
eral overview of the environments in which the inventors 
contemplate exemplary embodiments of the present in- 
vention will be used. 

Exemplary Embodiments 

[0037] Figure 3 Is a flow chart depicting a method 300 
for displaying a writing guide according to an exemplary 
embodiment of the present invention. In step 305, the 
writing guide module 227 determines whether a user 
has provided electronic ink by creating writing strokes 
on the tablet 250. If not, then the writing guide module 
227 repeats step 305 until it detects electronic ink. If the 
writing guide module 227 detects ink in step 305, then 
the method branches to step 310. 
[0038] In step 31 0, the writing guide module 227 clas- 
sifies the detected Ink as either a drawing or handwrit- 
ing. Based on that classification, the writing guide mod- 
ule 227 determines In step 315 whether the detected Ink 
comprises handwriting. If yes, then the method branch- 
es to step 320. In step 320, the writing guide module 
227 displays a handwriting writing guide (hereinafter 
"handwriting guide") on the tablet 250. In step 325, the 
writing guide module 227 continually detects additionai 
ink provided in the handwriting guide. As additionai ink 
is detected, the method proceeds to step 330, where the 
writing guide module 227 continuaily recaiculates the 
handwriting guide's dimensions to adjust to the size of 
the detected ink. The method then returns to step 325 
to continue detecting additional Ink In the handwriting 
guide. When the writing guide module 227 does not de- 
tect any additional ink in the handwriting guide, then the 
method branches to step 350. In step 350, the writing 
guide module 227 deactivates the handwriting guide to 
remove it from the tablet 250. In alternative embodi- 
ments of the present invention, the user can deactivate 
a handwriting guide by any of the following exemplary 
methods: (1) activation of another writing guide of any 
type; (2) hovering off the drawing guide for longer than 
one second or other predetermined time; and (3) switch- 
ing modes through the toolbar or other menu. Accord- 
ingly, any of those actions by the user can cause the 
writing guide module 227 to perform step 350 by deac- 
tivating the handwriting guide to remove it from the tablet 
250. 

[0039] Referring back to step 31 5, if the writing guide 
module 227 determines that the detected ink comprises 



a drawing, then the method branches to step 335. In 
step 335, the writing guide module 227 displays a draw- 
ing writing guide (hereinafter "drawing guide") on the 
tablet 250. In step 340, the writing guide module 227 
5 monitors the drawing guide to continually detect Ink In- 
put In the drawing guide. As the writing guide module 
227 detects Ink In the writing guide, the method contin- 
ually recalculates the drawing guide's dimensions in 
step 345 to adjust to the size of the detected ink. The 
10 method then returns to step 340 to continue detecting 
ink in the drawing guide. When the writing guide module 
227 does not detect additional ink in the drawing guide, 
the method branches to step 350. In step 350, the writing 
guide module 227 deactivates the drawing guide to re- 
's move it from the tablet 250. In alternative embodiments 
of the present invention, the user can deactivate a hand- 
writing guide by any of the following exemplary meth- 
ods: (1) activation of another writing guide of any type; 
(2) hovering off the drawing guide for longer than one 
20 second or other predetermined time; and (3) switching 
modes through the toolbar or other menu. Accordingly, 
any of those actions by the user can cause the writing 
guide module 227 to perform step 350 by deactivating 
the handwriting guide to remove it from the tablet 250. 
25 [0040] In an exempiary embodiment, a writing guide 
can be presented automatically when the user begins 
to write on the tablet 250 with the stylus 255. In an al- 
ternative exemplary embodiment, a writing guide can be 
presented initially by selecting a writing guide option 
30 from a menu. That selection can be made with a mouse, 
keyboard, or stylus. In another exemplary embodiment, 
the writing guide module 227 can recognize a gesture 
tap on a blank part of the tablet 250 as a way to insert 
a new writing guide. For example, the user can tap the 
35 styius 255 on a blank part of the tablet 255. Then, the 
writing guide module 227 can recognize the gesture tap 
and can insert a writing guide at that location. 
[0041] Figure 4 illustrates a handwriting guide 400 
displayed on a tablet 250 according to an exemplary em- 
40 bodiment of the present invention. The handwriting 
guide 400 comprises a writing area 402. As shown, a 
user can create handwriting 403 In the writing area 402 
with the stylus 255. The handwriting guide 400 compris- 
es other elements to enhance the available features. For 
45 example, the handwriting guide 400 comprises outline 
object handles 404, 406 to define the boundaries of an 
outline object. The handwriting guide 400 also compris- 
es an element node handle 408 to identify individual ob- 
ject elements within the outline object. For example, the 
50 Individual object eiements can comprise individual par- 
agraphs within the outline object. The element node 
handle 408 can allow moving or selecting individual ob- 
ject elements within the outline object. The handwriting 
guide 400 also comprises a merge handle 410 to allow 
55 merging of individual object elements. To merge individ- 
ual object elements, a user can click down on the merge 
handle 41 0 with the stylus 255 or a mouse and then can 
drag the writing guide 400 over other object elements. 
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[0042] Outline object handles 406 can operate ac- 
cording to the following exemplary embodiment. If a us- 
er has created an outline having three lines of handwrit- 
ing/text, then the lower right outline object handle 406 
is provided under the third line. Although the handle is 
under the third line, two more lines are available under 
that last line where the user can create new paragraphs. 
In an exemplary embodiment, the two additional, avail- 
able lines are not real until the user writes into them. 
The two additional lines can be accessed through the 
exemplary writing guide portions 2004a (Figure 20). If 
the user hovers the stylus 255 over either of the two new 
paragraph writing guide portions 2004a, the outline ob- 
ject handle 406 moves down underneath that line to 
show that writing on that writing guide portion will add 
the new paragraph to the already existing outline object. 
In an exemplary embodiment, the object handle 406 can 
remain in that extended location, even if the user moves 
the stylus 255 without inputting any handwriting into that 
line. 

[0043] The writing guide module 227 can default to 
single line paragraphs in the outline object. Accordingly, 
each new line of handwriting can be formatted as a sep- 
arate paragraph. For long sentences that span more 
than one line, the writing guide 400 can automatically 
extend to accommodate the additional space required. 
For example, as the user writes handwriting near the 
right end of the writing area 406, the writing area 406 
can expand to extend over the next line. Anything written 
into the next line then becomes part of the same para- 
graph as the previous line. In an exemplary embodi- 
ment, a boundary can be established near the right end 
of the writing area 406. Writing handwriting into the 
boundary area can extend the writing area over the next 
line. For example, the boundary area can equal three 
writing guides heights. 

[0044] In an alternative embodiment, the right bound- 
ary of the writing guide can be extended if the user con- 
tinues and writes past the edge of the right boundary. In 
that case, the writing guide gets wider. An extended writ- 
ing area on the next line can also be provided. Addition- 
ally, the extended writing area on the next line can re- 
main the original length or can be extended similarly to 
the line extended by the user. 

[0045] Additionally, multiple paragraphs can be 
merged manually into a single paragraph. For example, 

if the user begins the next line without writing handwrit- 
ing into the boundary area, then the next line comprises 
a separate paragraph. The separate paragraph exists 
even if the user intended the lines as part of the same 
paragraph. In that exemplary embodiment, the user can 
merge the two separate paragraphs into one paragraph. 
Each line (each object element) can have a merge han- 
dle 410 to indicate the separate object elements. Ac- 
cordingly the user can merge paragraphs or lines by 
dragging. one merge handle 410 over the other para- 
graph. 

[0046] Auseralsocan merge paragraphs by selecting 
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adjacent paragraphs and selecting a merge paragraphs 
option from a drop down menu. The user can make 
those selections using a mouse, keyboard, or stylus. In 
an alternative embodiment, the writing guide module 

5 227 can present a clickable user interface icon that hov- 
ers to the left of the handwriting guide. The user can 
click on the icon to merge two lines into one paragraph, 
or to split a paragraph into multiple paragraphs. 
[0047] In an exemplary embodiment, if the user starts 

'0 a new outline object close enough to the left boundary 
of the page such that the outline handle 404, node han- 
dle 408, orother various writing guides discussed below 
cannot be displayed on the viewing screen, then the out- 
line object can extend to the left off of the viewing 

15 screen. A horizontal page scroll bar can indicate the off- 
screen areas. 

[0048] Figure 5 is a flow chart depicting a method for 
classifying detected ink as a drawing or handwriting ac- 
cording to an exemplary embodiment of the present in- 
20 vention, as referred to in step 310 of Figure 3. In step 
505, an aspect ratio threshold can be established. In an 
exemplary embodiment, a user sets the aspect ratio 
threshold. Alternatively, a default aspect ratio setting 
can be used. The aspect ratio measures the length to 
25 height of ail detected ink and can provide an indication 
as to whether the detected ink comprises a drawing or 
handwriting. For example, handwriting typically com- 
prises ink having a high aspect ratio as a user writes 
along a line. A high aspect ratio corresponds to a large 
30 length when compared to the height. On the other hand, 
a drawing typically has a low aspect ratio due to the 
more uniform distribution of the ink. Such a uniform dis- 
tribution typically provides a length and height that are 
relatively close in size. 
35 [0049] In step 510, a number of minimum strokes can 
be established for classifying the detected ink as a hand- 
writing or drawing. Setting a minimum number of strokes 
can ensure detection of enough ink to provide an accu- 
rate classification. In an exemplary embodiment, the 
40 minimum number of strokes for handwriting can be three 
and for a drawing can be five. Other minimum numbers 
of strokes are within the scope of the present invention. 
In an alternative exemplary embodiment, a minimum 
amount of ink can be established for classifying the de- 
45 tected ink as a handwriting or drawing. 

[0050] In step 51 5, the writing guide module 227 can 
determine whether a user has input the minimum 
number of strokes for classifying the ink as handwriting. 
If not, then the method branches to step 520 to detect 
50 more ink strokes. If the user has input the minimum 
number of strokes for handwriting, then the method 
branches to steps 525 and 530. In steps 525 and 530, 
the writing guide module 227 determines the length and 
height, respectively, of the detected ink. In an alternative 
55 embodiment, step 515 can determine whether a user 
has input the minimum amount of ink for classifying the 
ink as handwriting. 

[0051] In step 535, the writing guide module 227 cal- 
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culates the aspect ratio for the detected ink, based on 
the length and height determined in steps 525 and 530, 
respectively. In step 540, the writing guide module 227 
determines whether the calculated aspect ratio Is great- 
er than the aspect ratio threshold set In step 505. If yes, 
then the writing guide module 227 determines in step 
545 that the detected ink is handwriting. The method 
then proceeds to step 315 (Figure 3). 
[0052] Referring back to step 540, If the writing guide 
module 227 determines that the calculated aspect ratio 
Is less than the threshold value, then the method 
branches to step 550. In an alternative embodiment, 
step 540 can determine If the writing guide has grown 
larger than twiee the height of a default line height. If so, 
then the method can branch to step 550 for classifying 
the Ink as a drawing. 

[0053] In Step 550, the writing guide module 227 de- 
termines whether the user has input the minimum 
number of strokes to classify the ink as a drawing. If not, 
then the method branches back to step 520 to detect 

more Ink strokes and to recalculate the aspect ratio of 
the detected Ink. If the user has Input the minimum 
number of strokes, then the method branches to step 
555 In which the writing guide module 227 determines 
that the detected ink Is a drawing. The method then pro- 
ceeds to step 315 (Figure 3). 

[0054] Figure 6 Is a flow chart depicting a method for 
displaying a handwriting guide according to an exem- 
plary embodiment of the present invention, as referred 
to in step 320 of Figure 3. The method of Figure 6 will 
be described with reference to Figures 7A and 7B, which 
illustrate various dimensions of exemplary handwriting 
guides. Figure 7A illustrates dimensions of a handwrit- 
ing guide 700a according to an exemplary embodiment 
of the present Invention. Figure 7B Illustrates dimen- 
sions of a handwriting guide 700b according to another 
exemplary embodiment of the present invention. In the 
exemplary embodiments, dimensions of the handwriting 
guide are based on a bounding rectangle defining the 
boundaries of the detected ink. 

[0055] In step 605 of Figure 6, the writing guide mod- 
ule 227 establishes a left edge 702 of the writing guide 
700a, 700b. The left edge 702 corresponds to the left 
edge of detected ink 701 . In step 61 0, the writing guide 
module 227 establishes a top edge 704 of the writing 
guide 700a, 700b. The top edge 704 corresponds to the 
top edge of the detected Ink 701 . In step 61 5, the writing 
guide module 227 establishes a baseline 706 of the writ- 
ing guide 700a, 700b. Exemplary embodiments for es- 
tablishing the baseline 706 are discussed below with ref- 
erence to Figures 8-11. In step 620, the writing guide 
module 227 establishes a right edge 708 of the writing 
guide 700a, 700b. The right edge 708 corresponds to 
the right edge of the detected ink 701 . In an alternative 
embodiment, the right edge 708 can be established 
based on a default outline width which the user can mod- 
ify. The right edge 708 can be adjusted to prevent it from 
extending past the viewing screen of the tablet 250 or 



overlapping another object on the page. 
[0056] Several alternatives exist for the final dimen- 
sions of the writing guide 700a, 700b. For example, the 
dimensions 702-708 established In steps 605-620 and 
5 Illustrated In Figure 7A can be used for the writing guide 
700a. Alternatively, those dimensions can be adjusted 
as shown in Figure 7B to provide a larger writing area 
for the writing guide 700b. Accordingly, In step 625, the 
writing guide module 227 determines whether to use the 
10 dimensions established In steps 605-620 for the writing 
guide. If yes, then the method branches to step 630. In 
step 630, the writing guide module 227 sets the dimen- 
sions of the writing guide 700a to the established dimen- 
sions. The method then proceeds to step 640. 
15 [0057] Referring back to step 625, If adjusted dimen- 
sions will be used for the handwriting guide, then the 
method branches to step 635. In step 635, the writing 
guide module 227 adjusts the dimensions of the writing 
guide. For example, as shown in Figure 7B, the writing 
20 guide module 227 can adjust the top edge 704 by a pre- 
determined spacing x to result in a new top edge 710. 
Additionally, the writing guide module 227 can adjust the 
baseline 706 by a predetermined spacing y to result in 
a new baseline 712. 
25 [0058] The predetermined spaclngs x and y can be 
chosen to provide an enlarged writing area for the writ- 
ing guide 700b. Accordingly, the predetermined spac- 
lngs X and y can allow for diacritics and comfort while 
writing. The amount of predetermined spacing x and y 
30 can be a function of a mathematical algorithm involving 
the dimensions of the bounding rectangle. However, 
other spacings are not beyond the scope of the present 
invention. For example, the spacings can be arbitrarily 
chosen instead of being the product of a calculation. In 
35 an exemplary embodiment, the predetermined spacing 
xcan be x = ten percent of the bounding rectangle height 
h (Figure 7A), and the predetermined spacing y can be 
y = twenty percent of the bounding rectangle height h. 
[0059] The method then proceeds to step 640. In step 
40 640, the writing guide module 227 determines whether 
the aspect ratio of the detected ink Is greater than a pre- 
determined threshold value. The aspect ratio provides 
a ratio of the length L to the height h (Figure 7A) or the 
height H (Figure 78) of the detected ink 701 . If not, then 
45 the method branches to step 650. If the aspect ratio is 
greater than the predetermined threshold value, then 
the method branches to step 645. In an exemplary em- 
bodiment, the predetermined aspect ratio threshold can 
be 3:1. Other aspect ratio thresholds are within the 
50 scope of the present invention. 

[0060] In step 645, the writing guide module 227 ad- 
justs the right edge 708 of the writing guide based on a 
default dimension. For example, the default dimension 
can comprise a right page margin or a right edge of the 
55 tablet's 250 viewing area. Accordingly, once the user 
has input a sufficient amount of ink to provide accurate 
dimensions for the writing guide, the writing guide can 
be extended across the viewing area. The method then 
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proceeds to step 650. 

[0061] In an exemplary embodiment, the right edge 
708 of the writing guide can extend to the right without 
overlapping an object or going off the viewing screen of 
the tablet 250. If the page width Is wider than the viewing 
screen, then the writing guide can extend to the edge of 
the viewing screen. 

[0062] In step 650, the writing guide module 227 ad- 
justs the dimensions of the writing guide to correspond 
to a font point size. For example, the dimensions of the 
writing guide can be adjusted to the nearest one-half 
point size. Accordingly, the dimension h (Figure 7A), or 
the dimension H (Figure 7B), can be adjusted to corre- 
spond to the nearest point size of the font. In that regard, 
the dimensions can be adjusted either up or down to 
correspond to the nearest point size. The method then 
proceeds to step 655 In which the writing guide module 
227 displays the writing guide. The method then pro- 
ceeds to step 325 (Figure 3). 

[0063] In an alternative exemplary embodiment, the 
writing guide dimensions can correspond to rule lines 
on the tabiet 250. For exampie, if the bottom edge of the 
caiculated writing guide is within a specified tolerance 
of the closest rule line, then the writing guide can be 
sized such that it's entire boundary is enclosed within 
the rule lines. In an exemplary embodiment, the speci- 
fied tolerance can be thirty percent. 
[0064] A method for establishing the baseline of a 
writing guide according to an exemplary embodiment of 
the present invention will now be described with refer- 
ence to Figures 8 and 9. Figure 8 is aflow chart depicting 
a method for establishing a baseline of a writing guide 
according to an exemplary embodiment of the present 
invention, as referred to in step 615 of Figure 6. Figure 
9 illustrates a representative ink histogram 904 showing 
the amount of ink in respective segments 902 of a line 
of ink 900. 

[0065] In step 805 of Figure 8, the writing guide mod- 
ule 227 establishes horizontal segments 902 for the line 
of ink 900. Any number of horizontal segments can be 
used for the line of ink 900. In an exemplary embodiment 
of the present invention, the writing guide module 227 
establishes thirty-two horizontal segments 902 for the 
line of ink 900. Other amounts of horizontal segments 
902 are within the scope of the present invention. 
[0066] In step 808, an ink threshold for removing a 
horizontal segment 902 from the writing area can be es- 
tablished. The segment removal threshold represents 
an amount of ink below which a segment 902 wiil be 
removed from the writing area. For example, bottom 
segments 902 of the iine of ink 900 typically will contain 
less ink. Accordingly, those segments can be removed 
from the writing area to determine the baseline of the 
writing guide. 

[0067] In an exemplary embodiment, the threshold for 
removing segments can be set at three percent. Accord- 
ingly, if a segment contains less than three percent of 
the line's ink, then that segment is removed from the 



writing guide dimensions. Other values for the threshold 
are within the scope of the present invention. 
[0068] In step 81 0, the writing guide module 227 de- 
termines whether It has detected Ink In the line of Ink 

5 900. If not, then the method repeats step 81 0 until Ink Is 
detected. When the writing guide module 227 detects 
ink in the line of ink 900, then the method branches to 
step 815. In step 815, the writing guide module 227 
measures the amount of ink in each horizontal segment 

• 0 902. in step 820, the writing guide module 227 creates 
a histogram 904 of the ink detected in each segment 
902. 

[0069] In step 825 the writing guide module 227 se- 
lects a segment 902 from the line of ink 900. 1 n step 830, 

'5 the writing guide module 227 determines whether the 
amount of Ink In the selected segment Is less than the 
predetermined segment removal threshold. If not, then 
the method branches to step 850 to select another seg- 
ment. If the ink amount of the selected segment is less 

20 than the predetermined threshold, then the method 
branches to step 835. 

[0070] In step 835, the writing guide module 227 de- 
termines whether the selected segment is the first or the 
last segment of the line of ink 900. The first and last seg- 
25 ments of the line of ink 900 correspond to the top and 
bottom segments, respectively, of the line of ink 900. If 
yes, then the method branches to step 840, where the 
writing guide module 227 removes the selected seg- 
ment from the line of ink 900. The method then proceeds 
30 to step 850 to select another segment. 

[0071] Referring back to step 835, if the selected seg- 
ment is not the first or last segment, then the method 
branches to step 845. In step 845, the writing guide mod- 
ule 227 determines whether the selected segment is ad- 
35 jacent to a segment previously removed from the line of 
ink 900. If not, then the method branches to step 850 to 
select another segment. If yes, then the method branch- 
es to step 840. In step 840, the writing guide module 
227 removes the selected segment from the line of ink 
40 900. The method then proceeds to step 850 to seiect 
another segment. Accordingly, only segments at the top 
or bottom of the line of ink 900 are removed. Segments 
having little ink and near the middle of the line of ink 900 
are not removed. 
45 [0072] In step 850, the writing guide module 227 de- 
termines whether to analyze another segment 902 of the 
line of ink 900. If yes, then the method branches back 
to step 825 to select another segment 902. If not, then 
the method branches to step 855. In step 855 the writing 
50 guide module 227 sets the baseline 706 (Figures 7A and 
7B) of the writing guide based on the lowest remaining 
segment 902 in the line of ink 900. The method then 
proceeds to step 620 (Figure 6). 

[0073] Additionally, the exemplary method of Figure 6 
55 can be used to set the top edge 704 of the writing guide. 
In that exemplary embodiment, the writing guide module 
227 sets the top edge 704 of the writing guide based on 
the highest remaining segment 902 in the line of ink 900. 
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[0074] In an exemplary embodiment, an initial com- 
pletion of step 825 comprises tine writing guide module 
227 selecting the lowest segment 902 in the line of ink 
900. Each repeat completion of step 825 comprises se- 
lecting the next highest segment 902 In the line of Ink 
900 until reaching a segment having an ink amount 
above the segment removal threshold. That segment 
then establishes the baseline 706 (Figures 7A and 7B) 
of the writing guide. The method then concludes without 
examining extraneous segments. 
[0075] Another method for establishing the baseline 
of a writing guide according to an alternative exemplary 
embodiment of the present invention will now be de- 
scribed with reference to Figures 10 and 11. Figure 10 
is a flowchart depicting a method for establishing the 
baseline of a writing guide according to an alternative 
exemplary embodiment of the present Invention, as re- 
ferred to in step 615 of Figure 6. To distinguish the al- 
ternative embodiments of step 61 5 described in Figures 
8 and 1 0, the embodiment of Figure 1 0 will be described 
as step 615'. Figure 11 illustratesdimensionsof awriting 
guide 1100 established by the method of Figure 10 ac- 
cording to an alternative exemplary embodiment of the 
present invention. 

[0076] In step 1005 of Figure 10, the writing guide 
module 227 determines whether ink strokes have been 
provided on the tablet 250. If not, then the method re- 
peats step 1005 until ink is detected. If yes, then the 
method branches to step 1010. In step 1010, the writing 
guide module 227 attempts to detect a word break be- 
tween words written on the tablet 250. If the writing gu ide 
module 227 cannot detect a word break in step 1010, 
then step 1010 is repeated. If the writing guide module 
227 detects a word break, then the method branches to 
step 1015. 

[0077] In step 1015, ink detected prior to the word 
break is sent to the text recognition module 235. In step 
1 020, the text recognition module calculates a base line 
1 1 02 and a midpoint 11 04 for the ink. Then, in step 1 025, 
the writing guide module 227 establishes the baseline 
706 (Figures 7A and 7B) of the writing guide at the cal- 
culated baseline 1102. 

[0078] I n step 1 030, the writing guide module 227 de- 
termines whether to calculate the top edge 704 (Figures 
7A and 7B) of the writing guide. If yes, then the method 
branches to step 1 035 in which the writing guide module 

227 calculates the top edge 1106 as twice the midpoint 
1 1 04. Accordingly, the writing guide module 227 estab- 
lishes the top edge 704 of the writing guide at the cal- 
culated top edge 1106. The method then branches to 
step 620 (Figure 6). 

[0079] Referring back to step 1030, if the writing guide 
module 227 will not calculate the top edge 704, then the 
method branches directly to step 620 (Figure 6). 
[0080] Figure 1 2 is a flowchart depicting a method for 
recalculating handwriting guide dimensions according 
to an exemplary embodiment of the present invention, 
as referred to in step 330 of Figure 3. In step 1205, a 



resizing ink threshold can be established for resizing the 
writing guide. In an exemplary embodiment, a user can 
establish the threshold for resizing the writing guide. Al- 
ternatively, a default threshold can be established for re- 
5 sizing the writing guide. The resizing Ink threshold can 
establish an ink amount outside of the writing guide be- 
yond which the writing guide expands to include the out- 
side ink. In one exemplary embodiment, the resizing ink 
threshold can be twenty percent. Accordingly, if twenty 
• 0 percent or more of the line's ink is outside of the current 
writing guide, then the writing guide extends up ordown, 
or both, until the outside ink is reduced below the thresh- 
old. Other resizing ink thresholds are within the scope 
of the present invention. 
15 [0081] Instep 1210, the writing guide module 227 es- 
tablishes horizontal segments on a writing area of the 
tablet 250. The writing area encompasses an area In- 
side and outside of the writing guide 400 (Figure 4). The 
horizontal segments can be established similarly to the 
20 horizontal segments 902 illustrated in Figure 9. In an ex- 
emplary embodiment, thirty-two segments can be es- 
tablished for the writing area. Other amounts of seg- 
ments are within the scope of the present invention. 
[0082] In step 1215, the writing guide module 227 
25 continually measures the amount of ink in each segment 
as ink strokes are provided on the tablet 250. In step 
1220, the writing guide module 227 creates an ink his- 
togram, tracking the amount of ink in each segment. The 
histogram can be similar to the histogram 904 illustrated 
30 in Figure 9. In step 1225, the writing guide module 227 
determines whether the amount of ink in all segments 
outside of the writing guide is greater than the resizing 
ink threshold. If not, then the method branches back to 
step 1215 to continue measuring the amount of ink in 
35 each segment as more ink strokes are provided on the 
tablet 250. 

[0083] If the writing guide module 227 determines in 
step 1 225 that the amount of ink in all segments outside 
the writing guide is greater than the resizing ink thresh- 

40 old, then the method branches to step 1230. In step 
1 230, the writing guide module 227 adjusts the top edge 
704 (Figures 7A and 78) and/or the baseline 706 (Fig- 
ures 7A and 78) until the amount of ink in all segments 
outside the writing guide is below the resizing threshold. 

45 In that regard, the top edge 704 can be reestablished at 
the top edge of all detected ink. Additionally, the base- 
line 706 can be reestablished as discussed above for 
step 615 of Figure 6. The method then proceeds to step 
325 (Figure 3), 

50 [0084] An exemplary embodiment, the writing guide 
can accommodate typical writing errors from the user. 
For example, the writing errors can include (1 ) lazy writ- 
ing where some words stray either partially or entirely 
into a line above or below the current line; (2) inflated 

55 writing where the user intentionally writes significantly 
larger so that a portion of the ink is on the writing guide, 
but a large portion is not; - and-(3) rebellious writing 
where the user continues to write past a fixed border of 
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the outline object. 

[0085] For lazy writing and inflated writing, the writing 
guide size can expand to encompass the ink outside of 
the initial writing guide dimensions. The expansion can 
be performed as described above with reference to Fig- 
ure 1 2. For rebeiiious writing, the right edge of the writing 
guide can be extended to aiiow the user to input addi- 
tional handwriting at the end of a line. The writing guide 
can be extended to the end of the viewing area on the 
tablet 250. Additionally, writing near the end of the writ- 
ing guide can trigger the extend paragraph writing guide 
onto the next available line (discussed above). 
[0086] Figure 1 3 illustrates a structured outline hand- 
writing guide 1300 for an indented outline structure ac- 
cording to an exemplary embodiment of the present in- 
vention. A writing guide 1300 can reveal indented out- 
line structure to the user and can provide a user inter- 
face on which to build the desired structure. 
[0087] As shown, the writing guide 1300 comprises 
the writing area 402. Within the writing area 402, the 
writing guide 1 300 comprises a next Indent level indica- 
tor 1 304, dividing a first level region 1 303 from a second 
level region 1 305. The indicator 1 304 illustrates the next 
indent level in an outline structure. If a user writes in the 
writing area 402 to the left of the indicator 1304, in the 
first level region 1 303, then the handwriting corresponds 
to handwriting on a first level of the outline structure. If 
a user writes in the writing area 402 to the right of the 
indicator 1304, in the second level region 1305, then the 
handwriting corresponds to handwriting on a second 
level of the outline structure. In an exemplary embodi- 
ment, the indent indicator 1 304 can disappear when ink 
enters the first level region 1303. 
[0088] The writing guide 1 300 also comprises a bullet 
region 1306. Before a user activates the buiiet region 
1 306, the bullet region 1 306 can present a lightiy shaded 
bullet or number. The lightly shaded bullet or number 
can indicate a bullet outline structure available to the 
user. If the user activates the buiiet region 1 306 with the 
stylus 255 or the pointing device 265, the lightly shaded 
bullet or number can be replaced by the real character. 
The user can activate the bullet region 1306 by tapping 
the stylus 255 on the bullet region 1306. Additionally, 
after activation, the user can select the bullet region 
1306 to edit the displayed number or to select another 
bullet type. 

[0089] In an exemplary embodiment, the bullet region 
1306 can have an initial default size to accommodate 
one or more numbers. The bullet region can grow to ac- 
commodate larger font size, writing guide height, or 
more numbers. For example, if the next number in the 
bullet series comprises three digits instead of two, then 
the bullet region widens to accommodate. 
[0090] Alternatively, a user can write ink into the bullet 
region 1306 to create a custom bullet. Any ink object 
input into the bullet region can create a custom bullet. 
A bullet region at the same level as a previous bullet 
region has the same ink symbol set as the custom bullet. 



Accordingly, selecting the bullet region inserts the cus- 
tom ink bullet. The bullet then can be repeated on sub- 
sequent lines. 

[0091] The writing guide 1300 also comprises a sec- 
5 ond level bullet region 1308. Hovering the stylus 255 
over the bullet region 1308 can produce a lightly shaded 
bullet or number in the bullet region 1308. Selecting the 
second level bullet region 1308 can activate the bullet 
as described above for the bullet region 1306. 
10 [0092] In an exemplary embodiment, the writing guide 
1300 can comprise multiple indent levels. Accordingly, 
the writing guide 1 300 can comprise multiple indent lev- 
el indicators to separate the multiple indent levels. Ad- 
ditionally, each indent level can have a corresponding 
15 bullet region to provide bullets for a respective indent 
level. 

[0093] In another exemplary embodiment, the writing 
guide 1300 can comprise features discussed above for 
the writing guide 400 of Figure 4. For example, the fea- 
20 tures can comprise a node handle, an outline object 
handle, and an object element handle. 
[0094] Figures 14A-14E illustrate an operation of the 
structured outline handwriting guide 1300 according to 
an exemplary embodiment of the present invention. Be- 
25 ginning in Figure 14A, a user has provided handwriting 
1402 on a first level of the structured outline. Addition- 
ally, the user has provided handwriting 1404 on a sec- 
ond level of the structured outline. As the user hovers 
the stylus 255 over the tablet 250, the structured outline 
30 handwriting guide 1300 appears. As shown in Figure 
1 4A, the user hovers the stylus 255 over the first level 
region 1303. Accordingly, the bullet region 1306 and the 
indicator 1304 are presented to guide the user in the 
structured outline. 
35 [0095] In Figure 14B, the user hovers the stylus 255 
over the second level bullet region 1308 to produce a 
iightiy shaded bullet or number for the second level re- 
gion 1 305. Thus, hovering the styius 255 within the sec- 
ond ievei bullet region 1308, to the left of the indent in- 
40 dicator 1304, causes the second level bullet writing 
guide to appear and moves the indent level of the writing 
guide to the next level. 

[0096] In Figure 14C, the user hovers the stylus 255 
over the second level region 1305. At that time, a third 

45 level indicator 1 406 is displayed to show the location of 
a third level region 1 407. A third level bullet region 1 408 
also is provided for the third level region 1407. 
[0097] In Figure 14D, the user hovers the stylus 255 
over the third level bullet region 1408 to produce a lightly 

50 shaded bullet or number for the third level region 1407. 
[0098] In Figure 14E, the user hovers the stylus 255 
over the third level region 1407 and can begin writing to 
provide text in the third level of the structured outline. 
[0099] Figure 15 illustrates a language handwriting 

55 guide 1500 for languages without spaces between their 
characters according to an exemplary embodiment of 
the present invention. The language handwriting guide 
1500 comprises the writing area 402. Additionally, the 
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language handwriting guide 1500 comprises character 
notches 1506 in the writing area 402. The character 
notches 1 506 provide a character box in the writing area 
402 into which a singie character from a particular lan- 
guage can be written. Accordingly, the character notch- 
es 1 506 divide the languages into their basic characters 
as the user writes the characters into separate character 
boxes. Accordingly, the handwriting guide 1500 can pro- 
vide word breal<ing for the text recognition module 235, 
thereby improving the accuracy of the text recognition 
process for languages without spaces between their 
characters. Japanese is an example of such a language. 
[0100] In an exemplary embodiment, the language 
handwriting guide 1 500 can be presented as discussed 
above with reference to Figure 3. Accordingly, a certain 
amount of ink may be provided on the tablet 250 before 
the full language handwriting guide is presented. Ink In- 
put before presentation of the language handwriting 
guide having the character notches can be marked as 
"word breaking needed." That marking can inform the 
text recognition module 235 that characters within the 
initial ink have not been separated. 
[0101] Figure 16 illustrates a gesture handwriting 
guide 1602 according to an exemplary embodiment of 
the present invention. As shown, the gesture handwrit- 
ing guide 1602 can be combined with the handwriting 
guide 400 described above. The gesture handwriting 
guide 1602 provides a holding area for one or more 
icons 1 604. Each icon 1 604 represents an action for ap- 
plication to handwriting provided in the writing area 402 
of the writing guide 400. For example, actions associat- 
ed with an icon 1604 can comprise highlight, bold, un- 
derline, and e-mail. Accordingly, selection of the icon 
1 604 can execute the action associated with that icon 
1 604. For example, selecting a highlight icon can cause 
all handwriting in the writing area 402 to become high- 
lighted. As another example, selecting an e-mail icon 
can cause the handwriting in the writing area 402 to be 
attached to an e-maii for sending to another party. 
[0102] An exemplary method 1700 for activating a 
gesture handwriting guide will now be described with 
reference to Figures 1 6 and 1 7. Figure 1 7 is a flow chart 
depicting the method 1 700 for activating agesture hand- 
writing guide according to an exemplary embodiment of 
the present invention. In step 1705, the writing guide 
module 227 attempts to detect ink strokes in an area to 
the left of the left margin. For example, as shown in Fig- 
ure 1 6, the writing guide module 227 can attempt to de- 
tect ink strokes in the gesture handwriting guide 1602. 
The writing can comprise a handwritten symbol corre- 
sponding to an icon 1604. If the writing guide module 
227 detects those inkstrokestothe left of the left margin, 
then the method branches to step 1715. In step 1715, 
the writing guide module 227 displays the icon corre- 
sponding to the handwritten symbol. The method then 
proceeds to step 1 720. 

[01 03] If the writing guide module 227 does not detect 
ink strokes to the left of the left margin in step 1 705, then 



the method branches to step 1710. In step 1710, the 
writing guide module 227 determines whether a user 
has selected an icon 1604 displayed in the gesture 
handwriting guide 1 602. If not, then the method branch- 

5 es back to step 1 705. Accordingly, steps 1 705 and 1710 
are repeated until the writing guide module 227 detects 
writing left of the margin or selection of an icon 1 604. 
[0104] If the writing guide module 227 determines in 
step 1710 that a user has selected an icon, then the 

<o method branches to step 1 720. In step 1 720, the writing 
guide module 227 determines an action attached to the 
icon. Then, in step 1725, the writing guide module 227 
performs the associated action. 

[0105] In an alternative exemplary embodiment, the 
gesture handwriting guide 1602 can be omitted. In that 
embodiment, a user can write a symbol anywhere on 
the tablet 250. The symbol can have a corresponding 
icon with an associated action. The writing guide module 
227 can recognize the symbol, display the Icon, and per- 
form the action, even though the user did not write the 
symbol in the gesture handwriting guide 1602. 
[0106] A method 1900 for activating a sacrificial writ- 
ing guide 1800 according to an exemplary embodiment 
of the present invention will now be described with ref- 
erence to Figures 18 and 19. Figure 18 illustrates the 
sacrificial writing guide 1 800 according to the exemplary 
embodiment of the present invention. Figure 19 is a flow 
chart depicting the method 1 900 for activating a sacrifi- 
cial writing guide 1800 according to an exemplary em- 
bodiment of the present invention. 
[0107] As shown in Figure 18, the sacrificial writing 
guide 1 800 comprises a sacrificial symbol 1 802 and the 
writing guide 400. In the exemplary embodiment of Fig- 
ure 18, the sacrificial symbol 1802 comprises a check 
box. Accordingly, a user can write the sacrificial symbol 
1802 on the tablet 250. The writing guide module 227 
can recognize the sacrificial symbol 1802 and can 
present the entire writing guide 1 800 for the user. 
[0108] In step 1905 of Figure 19, the writing guide 
module 227 attempts to detect a sacrificial symbol 1 802 
on the tablet 250. Step 1905 is repeated until the writing 
guide module 227 detects the-sacrificial symbol 1802. 
After detecting the sacrificial symbol 1802, the method 
branches to step 1910. In step 1910, the writing guide 
module 227 converts the sacrificial symbol 1 802 to text. 
Then, in step 1915, the writing guide module 227 dis- 
plays the complete writing guide 1 800 corresponding to 
the sacrificial symbol 1802. 

[0109] In the exemplary embodiment of Figure 18, the 
sacrificial symbol 1 802 comprises a check box. Accord- 
ingly, when the writing guide module 227 detects a 
handwritten symbol for a text box, the writing guide mod- 
ule 227 can display the complete writing guide 1800. 
[0110] Figure 20 illustrates active writing guide por- 
tions 2004, 2004a for adding handwriting to existing ob- 
ject elements according to an exemplary embodiment 
of the present invention. The writing guide portions 
2004, 2004a represent active areas on the tablet 250. 
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Each portion 2004 corresponds to an active area at the 
end of an existing, inactive object element 2002. For ex- 
ample, as shown in Figure 20, the inactive object ele- 
ment "windex" Is followed by an active writing guide por- 
tion 2004. If a user writes Ink Into the portion 2004, then 
the writing guide module 227 will add that Ink to the cor- 
responding object element 2002. 
[0111] Thus, the writing guide portions 2004 can allow 
a user to go back and write into a previous line. Any word 
written into a writing guide portion 2004 can be consid- 
ered on the corresponding writing guide and on the 
same line. Furthermore, the ink written into the writing 
guide portion 2004 can be entered before the next line 
in the stream of handwriting. 

[0112] Additionally, writing guide portions 2004a indi- 
cate an active area for adding a new object element to 
an outline object. Accordingly, a user can Input ink 
strokes Into the portions 2004a to add a new line of 
handwriting to the outline object. Thus, a new paragraph 
can be started by writing on a writing guide portion 
2004aof a blank line. In an exemplary embodiment, two 
free lines having writing guide portions 2004a are pro- 
vided below a line that contains text/handwriting. The 
writing guide portions 2004a can become visible by hov- 
ering the stylus 255 over them. The writing guide por- 
tions can acquire an initial height from the previous writ- 
ing guide in the same outline object. Alternatively, the 
writing guide portions can acquire the height of the first 
writing guide in the outline object if the previous writing 
guide had grown extensively. For example, if the previ- 
ous writing guide had grown to be twice the initial height, 
then the writing guide portions can acquire the height of 
the first writing guide. Additionally, if the first writing 
guide had grown extensively from the default height, 
such as twice the default height or larger, then the writing 
guide portions can acquire that size. 
[0113] Figure 20 also illustrates adding a new outline 
object 2006 on the tablet 250. To add the new outline 
object 2006, a user can start handwriting anywhere in 
blank space 2008 of the tablet 250. The blank space 
2008 can be any space that is not occupied by an inac- 
tive object element 2002 or a writing guide portion 2004, 
2004a. In an exemplary embodiment, outline objects 
can be overlapped. When a user inputs ink off of a writ- 
ing guide portion 2004 or a full writing guide line 2004a, 
the writing guide module 227 creates a new outline ob- 
ject 2006 that can float over an existing outline object. 
[0114] Figure 21 is a flow chart depicting a method 
21 00 for classifying off -guide writing according to an ex- 
emplary embodiment of the present invention. The 
method 21 00 allows the writing guide module to handle 
ink that is not considered structured handwriting writing 
or drawings. 

[0115] In step 21 05, the writing guide module 227 col- 
lects off-guide ink strokes provided on the tablet 250. 
Off guide ink stokes comprise any ink stroke that is not 
in an active portion 2004, 2004a (Figure 20). The writing 
guide module 227 can extract information from the ink 
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data as the off-guide ink strokes are collected in step 
21 05. For example, the data can comprise a time stamp 
and positional information for each ink stroke. The writ- 
ing guide module 227 then groups the Inkstrokes In step 

5 2110 based on the extracted information. For example, 
strokes can be grouped together If they are created with- 
in a predetermined time frame of each other, or within a 
predetermined distance from each other. Each subse- 
quent stroke meeting the above criteria can be added 

10 to the group. When the user inputs a stroke that fails the 
criteria, the writing guide module 227 begins a new 
group. 

[0116] In an exemplary embodiment, the predeter- 
mined time frame and distance can comprise three- 
's fourthsof a second and one-half inch, respectively. Oth- 
er time frames or distances are within the scope of the 
present Invention. 

[0117] In step 2115, the writing guide module 227 de- 
termines whether the size of the grouped strokes is less 
20 than a predetermined threshold size. Based on that de- 
termination, the writing guide module 227 can classify 
the off -guide ink as either an edit (step 2120) or an an- 
notation (step 2135). 

[0118] An ink edit is typically a few very short strokes, 
25 like crossing a "t" or dotting an "i." For example, Figure 
22A illustrates an ink stroke 2204 added to the object 
element 2202. The stroke 2204 comprises crossing the 
"t" In the object element 2202. Because the size of the 
stroke 2204 is below the threshold size, the writing guide 
30 module 227 classifies the stroke 2204 as an edit. In an 
exemplary embodiment, the threshold size can be the 
height h (Figure 7A) or H (Figure 7B) of the writing guide. 
Accordingly, if the off-guide ink stroke(s) fit into a box 
having a size of h by h (or H by H), then the stroke(s) 
35 comprise an edit. 

[0119] Thus, if the ink size is less than the predeter- 
mined threshold, then the writing guide module 227 
classifies the group of strokes as an edit in step 2120. 
In step 2125, the writing guide module 227 adds the 
to grouped ink strokes to the closest word object. Then, in 
step 2130, the writing guide module 227 marks the add- 
ed ink as "do not recognize." 

[0120] Referring back to step 2115, if the grouped ink 
strokes exceed the predetermined threshold size, then 
45 the method branches to step 2135. In step 2135, the 
writing guide module 227 classifies the grouped ink 
strokes as an annotation. 

[0121] Accordingly, the writing guide module 227 can 
classify off-guide ink having a size larger than the pre- 

50 determined threshold as an annotation. After receiving 
enough ink to classify the ink group as an annotation, 
the writing guide module 227 can provide a drawing writ- 
ing guide. Thus, in step 2140, the writing guide module 
227 creates a drawing for the grouped strokes. The 

55 drawing writing guide can float over the underlying out- 
line object. For example. Figure 22B illustrates a group 
of off-gu ide 1 nk strokes 2208 rep resenti ng an an notation 
according to an exemplary embodiment of the present 



25 



30 



14 



27 EP 1 376 390 A2 28 



invention. Tlie writing guide module classifies the ink 
strokes 2208 of Figure 22B as an annotation. Tfie ink 
strokes 2208 can comprise a drawing floating over tfie 
outline object 2206. In an exemplary embodiment, the 
annotation can be anchored to the node or object that it 
Is written over. Accordingly, the Items can move around 
on the page together. 

[0122] Figures 23A and 23B illustrate a nested head- 
ing writing guide 2300 according to an exemplary em- 
bodiment of the present invention. After a writing guide 
2300 has been extended to full length, and the user has 
written handwriting into it, an indicator 2304 appears to 
indicate the beginning of a nested heading. As shown 
in Figure 23A, the indicator 2304 divides the writing 
guide 2300 into a first heading region 2302 and a second 
heading region 2306. Writing Into the first heading re- 
gion 2302 places ink Into that region. As shown In Figure 
23B, writing Into the second heading region 2304 places 
Ink Into that region, thereby creating the nested head- 
ings. 

[0123] Figure 24 illustrates a drawing writing guide 
2400 according to an exemplary embodiment of the 
present invention. As shown, the drawing guide 2400 
can be created for a group of ink strokes 241 0 provided 
on the tablet 250. The drawing symbol 2403 of the draw- 
ing guide 2400 indicates that the writing guide module 
227 classified the ink strokes 2410 as a drawing. 
[0124] The drawing guide 2400 also comprises resiz- 
ing handles 2406. The resizing handles 2406 allow man- 
ual resizing of the drawing guide 2400. A user can resize 
the drawing guide 2400 by selecting and dragging one 
of the drawing guide resizing handles 2406. The draw- 
ing guide 2400 also comprises a move handle 2404 for 
relocating the entire drawing guide 2400. 
[0125] A drawing tools icon 2408 provides access to 
a drop down menu (not shown). The drop down menu 
can comprise options for reclassifying the ink or for lock- 
ing the drawing guide. Accordingly, the user can manu- 
ally change the ink classification for a drawing or hand- 
writing. For example, the user can select the drawing 
tools Icon 2408 to reveal the drop down menu (not 
shown). The user then can select a "treat as writing" op- 
tion to change the drawing classification to a handwriting 
classification. Additionally, the user can lock a drawing 
to prevent interaction with other drawings. Once locked, 
the drawing behaves like a regular picture such as a bit- 
map or jpeg picture file. A locked drawing guide cannot 
be merged with another drawing. To lock a drawing, the 
user can select the drawing tools icon 2408 to reveal the 
drop down menu (not shown). The user then can select 
a "lock drawing option." 

[0126] Figure 25 illustrates a method for displaying a 
drawing guide 2400 according to an exemplary embod- 
iment of the present invention, as referred to step 335 
of Figure 3. In step 2505, the writing guide module 227 
establishes a bounding rectangle 2412 around the de- 
tected Ink strokes 2410. The bounding rectangle 2412 
comprises a rectangle encompassing the ink strokes 



2410. In step 2510, the writing guide module 227 adds 
a hot zone 2414 around the bounding rectangle 2412. 
The hot zone 2414 provides additional space in the 
drawing guide 2400 for a user to Input new Ink strokes. 

5 [0127] In an exemplary embodiment, the hot zone 
2414 can comprise a fixed boundary around the bound- 
ing rectangle 2412. In an alternative embodiment, the 
hot zone 241 4 can comprise a buffer of a specified width 
surrounding each individual stroke of the detected 

• 0 strokes 241 0. In that case, then the hot zone can have 
an irregular shape. 

[0128] Finally, in step 2515, the writing guide module 
227 displays the drawing guide 2400 having the dimen- 
sions described above on the tablet 260. The method 
'5 then proceeds to step 340 (Figure 3). 

[0129] In an alternative embodiment, a drawing writ- 
ing guide can be inserted initially by selecting a drawing 
guide from a menu. The user can make that selection 
using a mouse, keyboard, or stylus. 
[0130] The user can deactivate a drawing guide by the 
following exemplary methods: (1) activation of another 
writing guide of any type; (2) hovering off the drawing 
guide for longer than one second or other predeter- 
mined time; and (3) switching modes through the toolbar 
or other menu. 
[0131] Figure 26 is a flowchart depicting a method for 
recalculating drawing guide dimensions according to an 
exemplary embodiment of the present Invention, as re- 
ferred to In step 345 of Figure 3. In step 2605, the writing 
guide module 227 attempts to detect Ink outside of the 
bounding rectangle 2412 of the drawing guide 2400. 
Step 2605 can be repeated until the writing guide mod- 
ule 227 detects ink outside of the bounding rectangle 
2412. Once ink outside of the bounding rectangle 2412 
is detected, the method branches to step 2610. In step 
2610, the writing guide module 227 reestablishes the 
bounding rectangle's 2412 dimensions to encompass 
the newly detected ink. In step 2615, the writing guide 
module 227 adds a new hot zone 2414 around the re- 
established bounding rectangle 2412. Then, in step 
2620, the writing guide module 227 displays the resized 
drawing guide 2400. The method then proceeds to step 
340 (Figure 3). 

[0132] In an exemplary embodiment, the drawing 
guide can be extended by drawing a stroke that begins 
on the drawing guide and extends off drawing guide. Al- 
ternatively, the drawing guide can be extended by draw- 
ing a stroke that begins off the drawing guide and ex- 
tends into a hot zone of the drawing guide. In either 
case, the drawing guide can be extended so that the 
stroke is entirely contained within the drawing guide. 
[0133] In another exemplary embodiment, a user can 
add ink to a drawing guide even though the ink is not 
input into the drawing guide. In that case, the drawing 
guide can expand to encompass the added ink. For ex- 
ample, the writing guide module 227 can determine 
whether the added ink belongs to the writing guide 
based upon the time stamp and position of each added 
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stroke. For instance, the following ink strokes can be 
considered part of tine drawing, tfiereby causing tfie 
drawing guide to expand and encompass them: (1) a 
stroke input within one-half inch and three-fourths of a 
second of another stroke in the drawing guide; (2) a 
stroke input within two inches and one-half second of 
another stroke in the drawing guide; and (3) a stroke 
input within five inches and one-quarter second of an- 
other stroke in the drawing guide. Other time and posi- 
tion amounts are within the scope of the present inven- 
tion. 

[0134] In another exemplary embodiment, two or 
more drawing guides can be merged together to form 
one larger drawing guide. For example, if an ink stroke 
from one drawing guide crosses into a hot zone of an- 
other drawing guide, then the two drawings guides can 
be merged together. The resulting drawing guide can 
have the size of the bounding rectangle of the two orig- 
inal drawing writing guides plus a new hot zone. Merging 
also can be accomplished by selecting the drawing 
guides and selecting a merge option from a menu. 
[0135] The present invention can be used with com- 
puter hardware and software that performs the methods 
and processing functions described above. As wili be 
appreciated by those skilled in the art, the systems, 
methods, and procedures described herein can be em- 
bodied in a programmable computer, computer execut- 
able software, or digital circuitry. The software can be 
stored on computer readable media. For example, com- 
puter readable media can include a floppy disk, RAIVI, 
ROM, hard disk, removable media, flash memory, mem- 
ory stick, optical media, magneto-optical media, 
CD-ROM, etc. Digital circuitry can include integrated cir- 
cuits, gate arrays, building block logic, field programma- 
ble gate arrays (FPGA), etc. 

[0136] Although specific embodiments of the present 
invention have been described above in detail, the de- 
scription is merely for purposes of illustration. Various 
modifications of, and equivalent steps corresponding to, 
the disclosed aspects of the exemplary embodiments, 
in addition to those described above, can be made by 
those skilled in the art without departing from the spirit 
and scope of the present invention defined in the follow- 
ing claims, the scope of which is to be accorded the 
broadest interpretation so as to encompass such mod- 
ifications and equivalent structures. 



Claims 

1. A computer-implemented method for displaying a 
writing guide for a free-form document editor, com- 
prising the steps of: 

detecting electronic ink comprising an ink 
stroke; 

classifying the detected ink as corresponding 
to one of handwriting and a drawing; and 
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displaying a handwriting writing guide in re- 
sponse to classifying the detected ink as hand- 
writing. 

5 2. The method according to Claim 1, wherein said 
classifying step comprises the steps of: 

setting a threshold aspect ratio indicating 
whether the detected Ink comprises handwrit- 
10 ing or a drawing; 

determining a length and height of the detected 
ink; 

computing an ink aspect ratio for the detected 
ink based on the determined length and height; 
'5 determining whether the ink aspect ratio is 

greater than the threshold aspect ratio; and 
classifying the detected Ink as handwriting in 
response to a determination that the ink aspect 
ratio Is greater than the threshold aspect ratio. 

20 

3. The method according to Claim 2, further compris- 
ing the steps of: 

setting a minimum stroke threshold corre- 
25 spending to a predetermined number of ink 

strokes to detect in said detecting step before 
classifying the detected ink as handwriting or a 
drawing; 

determining a detected number of Ink strokes 
30 for the detected Ink; and 

determining whether the detected number of 
ink strokes is greater than the minimum stroke 
threshold, 

35 wherein said classifying step is performed in 

response to a determination that the detected 
number of ink strokes is greater than the minimum 
stroke threshold. 

40 4. The method according to Claim 3, wherein the min- 
imum stroke threshold comprises a predetermined 
number of ink strokes for classifying the detected 
ink as handwriting and a different predetermined 
number of ink strokes for classifying the detected 

45 ink as a drawing. 

5. The method according to Claim 1, wherein said 
handwriting writing guide comprises a writing area 
into which a user inputs the electronic ink. 

50 

6. The method according to Claim 5, wherein said dis- 
playing step comprises the step of determining di- 
mensions of the writing area based on a bounding 
rectangle that encompasses the detected ink. 

55 

7. The method according to Claim 6, wherein said step 
of determining dimensions comprises the step of 
determining a right, left, top, and baseline dimen- 
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sion of the writing area based on the bounding rec- 
tangle that encompasses the detected Ink. 

8. The method according to Claim 7, wherein said step 
of determining dimensions further comprises the 
step of Increasing the writing area by adding prede- 
termined spaclngs to one of the top and baseline 

dimensions. 

9. The method according to Claim 6, wherein said step 
of determining dimensions comprises the steps of: 

creating a histogram representing a distribution 
of the detected inl< on a line; and 
calculating the baseline dimension of the writ- 
ing area based on the histogram. 

10. The method according to Claim 9, wherein said cre- 
ating step comprises the steps of: 

dividing the line into a plurality of segments; 
measuring an amount of the detected ink in 
each segment; and 

creating the histogram based on the amount of 
detected ink in each segment. 

11. The method according to Claim 10, wherein said 
calculating step comprises the steps of: 

setting an Ink removal threshold corresponding 
to an amount of Ink below which a segment Is 
removed from the writing area; 
determining whether the amount of detected 

ink in a segment is less than the ink removal 

threshold; and 

removing the segment from the writing area to 
establish the baseline dimension of the writing 
area in response to a determination that the 
amount of detected ink in the segment is less 
that the ink removal threshold. 

12. The method according to Claim 6, wherein said step 
of determining dimensions comprises the steps of: 

converting a portion of the detected Ink to text; 
and 

calculating a baseline dimension of the writing 
area based on the Ink-to-text conversion of said 
converting step. 

13. The method according to Claim 6, further compris- 
ing the steps of: 
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determining whether the amount of ink outside 
the writing area is greater than the resizing 
threshold; and 

resizing the writing area dimensions In re- 
5 sponse to a determination that the amount of 

Ink outside the writing area Is greater than the 
resizing threshold. 

14. The method according to Claim 6, further comprls- 
10 ing the steps of: 

setting a resizing threshold corresponding to an 
amount of Ink outside the writing area for resiz- 
ing the dimensions of the writing area; 

'5 establishing a plurality of segments Inside and 

outside the writing area; 
detecting an amount of Ink In each segment; 
determining whether the amount of Ink In seg- 
ments outside the writing area Is greater than 

20 the resizing threshold; and 

Increasing the writing area dimensions In re- 
sponse to a determination that the amount of 
ink outside the writing area Is greater than the 
resizing threshold. 

25 

15. A computer-implemented method for displaying a 
writing guide for a free-form document editor, com- 
prising the steps of: 

30 detecting electronic Ink; 

classifying the detected ink as corresponding 
to one of handwriting and a drawing; and 
displaying a handwriting writing guide compris- 
ing a writing area Into which a user Inputs the 

35 electronic Ink, In response to classifying the de- 

tected Ink as handwriting. 

16. The method according to Claim 15, wherein said 
displaying step comprises displaying the handwrit- 

40 ing writing guide further comprising a next level in- 
dent Indicator dividing the writing area Into a first 
level region and a second level region, the first and 
second level regions corresponding to different lev- 
els of a structured outline. 

45 

17. The method according to Claim 16, wherein said 
displaying step comprises displaying the handwrit- 
ing writing guide further comprising a bullet region 
corresponding to bullet outline of the structured out- 

50 line. 

18. The method according to Claim 17, further compris- 
ing the step of activating the bullet region to display 
the bullet outline. 
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setting a resizing threshold corresponding to an 
amount of Ink outside the writing area for resiz- 
ing the dimensions of the writing area; 
detecting an amount of Ink outside the writing 
area; 



55 

19. The method according to Claim 15, wherein said 
displaying step comprises displaying the handwrit- 
ing writing guide further comprising character 
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notches that divide the writing area into character in response to classifying the detected ink as hand- 

boxes for individual characters of a language. writing. 

20. The method according to Claim 15, wherein said 
handwriting writing guide comprises a gesture 5 
handwriting writing guide. 

21. The method according to Claim 15, wherein said 
handwriting writing guide comprises a sacrificial 
handwriting writing guide. io 

22. The method according to Claim 15, wherein said 
handwriting writing guide comprises a nested head- 
ing handwriting writing guide. 

15 

23. A computer-implemented method for displaying a 
writing guide for a free-form document editor, com- 
prising the steps of: 

detecting electronic ink; 20 
classifying the detected ink as corresponding 
to one of handwriting and a drawing; and 
displaying a drawing guide in response to clas- 
sifying the detected ink as a drawing. 

25 

24. The method according to Claim 23, wherein said 
displaying step comprises the steps of: 

determining dimensions of an original bounding 
rectangle that encompasses the detected Ink; 30 
and 

establishing dimensions of the drawing guide 
based on the original bounding rectangle di- 
mensions. 

35 

25. The method according to Claim 24, further compris- 
ing the steps of: 

detecting ink outside the original bounding rec- 
tangle; 40 
determining dimensions of a new bounding rec- 
tangle that encompasses the ink detected out- 
side the original bounding rectangle; and 
reestablishing the dimensions of the drawing 
guide based on the new bounding rectangle dl- 
menslons. 

26. The method according to Claim 24, wherein said 
displaying step further comprises determining di- 
mensions of a hot zone around the originai bound- so 
ing rectangle, and 

wherein said establishing step comprises es- 
tablishing the dimensions of the drawing guide 
based on the bounding rectangle and hot zone di- 
mensions. 55 

27. The method according to Claim 24, further compris- 
ing the step of displaying a handwriting writing guide 
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Figure 3 
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Set aspect ratio threshold 



Set minimum strolces value 
for classifying as liandwriting 
or drawing 
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detected ink is a 
drawing 
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Figure 5 
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Establish left edge of the 
writing guide at the left edge 
of the detected ink 



Establish top edge of the 
writing guide at the top edge 
of the detected ink 



Establish baseline of the 
writing guide 



Establish right edge of the 
writing guide at the right edge 
of the detected ink 




Setthewritin 


9 guide 
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Adjust the right edge of 
the writing guide based 
on a default dimension 



Adjust writing guide 
dimensions to 
correspond to a font point 
size 
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Display the writing guide 




Figure 6 
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Figure 9 
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Figure 1 0 
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Set ink threshold for resizing 
the writing guide 
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Establish horizontal 
segments for the writing area 
(inside and outside the 
writing guide) 
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each segment 
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Create histogram tracking the 
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Figure 12 
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Figur0 14A 
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Figure 16 
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Figure 19 
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Figure 20 
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Figure 21 
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Figure 22A 
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Figure 25 
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Figure 26 



